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(57) In editing first and second compressed pic- 
ture data each having been obtained by predic- 
tion-coding, the second compressed picture 
data is modified by decoding a frame of the 
second compressed picture data at an editing 
point and re-coding the decoded data so that 
every frame around the editing point can be 
decoded after editing. The first compressed 
picture data and the modified second compres- 
sed picture data are liked to complete the edit- 
ing. In mixing a compressed picture data with a 
non-compressed picture data, the compressed 
picture data is decoded and added to the non- 
compressed picture data to obtain a mixed 
picture data. The mixed picture data is re-coded 
to be a compressed picture data In each of the 
editing and mixing, the re-coding may be per- 
formed by using at least one of the motion 
compensation information, motion vector infor- 
mation and orthogonal transform mode infor- 
mation which can be derived from the 
compressed picture data in the decoding pro- 
cess. 
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BACKGROUND OF THE INVENTION 

1. Field of the Invention 

The present invention relates to a compressed 5 
picture editing method for editing compressed picture 
data, and a picture coding apparatus for mixing a part 
or whole of compressed picture data with other com- 
pressed picture data or real time picture data. 

10 

2. Description of the Prior Art 

A digital video signal possesses an enormous 
amount of information, and picture coding is indis- 
pensable for transmission and recording. Various pic- 15 
ture coding techniques have been developed recent- 
ly, and some of them are produced as encoders and 
decoders. 

As an example, a conventional picture coding ap- 
paratus using the MPEG (moving picture coding ex- 20 
perts group) method is described below while refer- 
ring to accompanying drawings. 

Fig. 1 5 is a block diagram of the conventional pic- 
ture coding apparatus. The apparatus comprises a 
motion detector 81 , a DCT (discrete cosine transform) 25 
mode judging circuit 82, a DCT circuit 83, a quantizing 
circuit 84, a variable length coding circuit 85, an in- 
verse quantizing circuit 86, an inverse DCT circuit 87, 
a frame memory 88, and a motion compensation cir- 
cuit 89. Fig. 16 is an explanatory diagram of motion 30 
compensation predicting method, and Fig. 17 is an 
explanatory diagram of the frame memory 88 and the 
motion compensation circuit 89. 

In thus constituted conventional picture coding 
apparatus, a video signal is scanned by interlacing, 35 
and is entered as being divided into frame units. A pic- 
ture in a first frame to be coded, that is, frame t in Fig. 
1 6 is processed by intraf rame coding using the data 
within the frame. First, the DCT mode judging circuit 
82 detects the motion of the input picture data in each 40 
two-dimensional block of pixels by, for example, cal- 
culating inter-line differences, determines from the 
detection result whether to perform DCT in frame unit 
or in field unit, and outputs the determination result as 
DCT mode information. The DCT circuit 83 receives 45 
the DCT mode information, and performs the DCT in 
either frame unit or field unit to transform the picture 
data into transform coefficients. The transform coef- 
ficients are quantized in the quantizing circuit 84, and 
variable-length coded in the variable length coding 50 
circuit 85 to obtain a coded bit stream which is sent 
out to a transmission line. The quantized transform 
coefficients are simultaneously fed into the inverse 
quantizing circuit 86 and the inverse DCT circuit 87 to 
be returned to real time data, and stored in the frame 55 
memory 88. 

Generally, pictures are high in degree of correla- 
tion, and DCT causes to concentrate the energy on 



the transform coefficients corresponding to low fre- 
quency components. Therefore, by quantizing rough- 
ly the high frequency components which are less ob- 
vious visually and finely the low frequency compo- 
nents which are important components, the picture 
quality deterioration is kept to a minimum, and the 
data quantity can be decreased at the same time. In 
the interlace scanned picture, if the motion is small, 
the intraf rame correlation is strong, and if the motion 
is large, the interframe correlation is small while the 
intra-f ield correlation is high. By making use of such 
characteristics of interlace scanning, i.e., by changing 
over the frame-based DCT and the field-based DCT, 
the interlaced picture can be coded efficiently. 

On the other hand, in the pictures afterf rame t+1 , 
predicted picture values are calculated in every 
frame, and differences of the predicted picture values 
from the original picture values, i.e., the predicted er- 
rors are coded. 

In MPEG, the calculating method of the predicted 
picture value includes forward prediction, backward 
prediction, and bidirectional prediction. Fig. 16 is an 
explanatory diagram of the prediction methods. The 
frame at time t is intraframe coded (hereinafter, an in- 
traframe coded frame is called I frame). Then, a dif- 
ference of the frame at time t+3 from a frame ob- 
tained by decoding the I frame is calculated after mo- 
tion compensation, and this difference is coded. This 
operation for predicting a frame which is ahead in 
time is called forward prediction (hereinafter, a frame 
coded by forward prediction is called P frame). 
Frames at time t+1 , t+2 are similarly subjected to mo- 
tion compensation, difference calculation and differ- 
ence coding by using frames decoded from the I and 
P frames. At this time, the predicted picture is com- 
posed by selecting in a block having a minimum error 
from the blocks of I frame (forward prediction), P 
frame (backward prediction), and the mean of I frame 
and P frame (bidirectional prediction) (hereinafter, a 
frame coded at a part or whole thereof by bidirectional 
prediction is called B frame). The B frame is predicted 
from frames before and after in time, so that a newly 
appearing object can be predicted accurately, thereby 
enhancing the coding efficiency. 

As the encoder, in the first place, the motion vec- 
tor to be used in prediction is detected in the motion 
detector 81 on a two-dimensional block by block basis 
by, for example, the well-known full search method. 
Next using the detected motion vector, the frame 
memory 88 and the motion compensation circuit 89 
generate a next predicted value compensated for the 
motion in each two-dimensional block. 

Fig. 1 7 is an example of constitution of the motion 
compensation circuit 89. Herein is explained genera- 
tion of predicted value of bidirectional prediction. The 
motion vector calculated in the motion detector 81 is 
fed into an address circuit 882 in the frame memory 
88, and pictures of I and P frames stored in the frame 



2 



3 



EP 0 656 729 A2 



4 



memory 881 are read out At this time, to correspond 
to the interlaced picture same as in DCT, the two-di- 
mensional blocks are formed in each frame or each 
field, and the vectors and predicted picture values are 
generated respectively. In each two-dimensional 5 
block, as prediction errors, forward prediction error, 
bidirectional prediction error and backward prediction 
error by using frame vectors, and forward prediction 
error, bidirectional prediction error and backward pre- 
diction error by using field vectors, that is, six kinds 10 
of errors in total are calculated in square error calcu- 
lating circuits 893 to 898. A smallest one of the six er- 
rors is selected in an error comparator 899, and the 
predicted values and prediction mode information are 
issued. The above prediction mode information, mo- »5 
tion vector, and DCT mode information are variable- 
length coded in the variable length coding circuit 85, 
and sent out to the transmission line together with the 
DCT transform coefficients. 

Thus, according to such a picture coding appara- 20 
tus, since the predicted error is optimally coded, the 
energy is decreased and coding of higher efficiency 
is realized as compared with the case of direct coding 
of picture data such as intraframe coding. (Refer, for 
example, ISO/IEC JTC1/SC29/WG11 N0502, "Gen- 25 
eric Coding of Moving Pictures and Associated Au- 
dio." July 1993.) 

However, when editing the compressed picture 
data coded in such picture coding method, various 
problems occur because the differences of picture 30 
data are coded. Fig. 18 is an explanatory diagram 
showing a conventional editing method of com- 
pressed picture data. Referring now to Fig. 18, the 
problems are explained below. In Fig. 18, suppose to 
link compressed picture data of (a) and (b) at the dot- 35 
ted line portions. Numerals given in Fig. 18 represent 
frame numbers. Since the B frame is coded after the 
I and P frames have been coded, the sequence of the 
coded frames is changed from the sequence to be 
displayed. After linking the compressed picture data 40 
of (a) and (b), the P and B frames right after the edit- 
ing point, that is, the P frame of frame number 8 and 
the B frames of frame numbers 6 and 7 in the com- 
pressed picture data (b) cannot be decoded because 
the I frame of frame number 5 used in prediction is 45 
lost Also, the pictures after frame 9 used in prediction 
of the P frame of frame number 11 cannot be decod- 
ed. 

Moreover, various problems occur when the com- 
pressed picture data coded by the picture coding so 
method with other picture data. The conventional pic- 
ture mixing of analog signals is realized by adding ele- 
ments together, but the compressed picture data are 
coded in variable length, and cannot be added simply 
on a bit by bit basis. Besides, in order to perform mix- 55 
ing by decoding the compressed picture to restore 
real time picture data, adding the decoded picture to 
other picture and coding the added result again, the 



mixing apparatus must be provided with both a de- 
coder and an encoder, which results in a large size. 

SUMMARY OF THE INVENTION 

It is hence a primary object of the invention to 
present an editing and mixing methods of com- 
pressed pictures, and an editing and mixing appara- 
tuses of compressed pictures, capable of editing and 
mixing compressed pictures compression-coded by 
the interf rame difference coding in a simple constitu- 
tion. 

To achieve the object, the invention presents a 
compressed picture editing method comprising the 
steps of: receiving first and second compressed pic- 
ture data each having been obtained by prediction- 
coding by at least one type of prediction out of a for- 
ward prediction for predicting a picture data in a cur- 
rent frame from a previous frame ahead in time, a 
backward prediction for predicting from a subsequent 
frame behind in time, and a bidirectional prediction si- 
multaneously using both the forward prediction and 
the backward prediction; decoding the second com- 
pressed picture data; intraframe coding a frame of 
the decoded picture corresponding to a editing point; 
prediction-coding again a part or whole of the other 
frames of the decoded picture by using motion com- 
pensation, motion vector, and orthogonal transform 
mode information obtained by decoding the second 
compressed picture data to obtain modified second 
compressed picture data; and linking the first com- 
pressed picture data with the modified second com- 
pressed picture data. 

In this constitution, the compressed picture data 
is once decoded, and the frame right after the editing 
point is coded again into an I frame, so that the pre- 
dicted picture will not be lost by editing. For coding P 
and B frames, the motion compensation, motion vec- 
tor and DCT mode information obtained by decoding 
are used, so that a motion detector and a DCT mode 
judging circuit required for mass calculation in the 
conventional picture encoder are not necessary, and 
the motion compensation circuit is simplified. Accord- 
ingly, the compressed picture data can be edited in a 
simple constitution. 

In a compressed picture mixing method of the in- 
vention, a first compressed picture data is decoded, 
the decoded picture data and a second picture data 
are added to create a mixed picture data, and the 
mixed picture data is prediction-coded again by using 
motion compensation information, motion vector, and 
orthogonal transform mode information obtained by 
decoding the first compressed picture data. 

In another method, a first compressed picture 
data and a second compressed picture data having 
been compression-coded in the same manner as in 
the first compressed picture data are decoded, ortho- 
gonal transform coefficients of the first compressed 
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picture data and the second compressed picture data 
are added to create a mixed picture data, and the 
mixed picture data is prediction-coded again by using 
motion compensation information, motion vector, and 
orthogonal transform mode information obtained by 
decoding the first compressed picture data. 

According to this constitution, since the real time 
picture data or orthogonal transform coefficients are 
added after decoding the compressed picture data, 
even compressed picture data coded by variable 
length coding can be mixed. In addition, when coding 
the mixed picture again, since the motion compensa- 
tion, motion vector, and orthogonal transform mode 
information obtained by decoding are used, a motion 
detector and an orthogonal transform mode judging 
circuit required for mass calculation in the convention- 
al picture encoder are not necessary, and the motion 
compensation circuit is simplified. Accordingly, the 
compressed picture data can be mixed in a simple 
constitution. Furthermore, the picture data to be 
mixed may be multiplexed on the compressed picture 
data in transmission, so that a picture mixing appara- 
tus light in the load at the coding side can be realized. 

BRIEF DESCRIPTION OF THE DRAWINGS 

Fig. 1 is a block diagram of a compressed picture 
encoder in a first embodiment of the invention. 

Fig. 2 is a constitution diagram of a simplified mo- 
tion compensation circuit and a frame memory in the 
first embodiment of the invention. 

Fig. 3 is an explanatory diagram of algorithm 
showing an example of operation of CPU in the first 
embodiment of the invention. 

Fig. 4 is an explanatory diagram showing an edit- 
ing method of compressed picture data in the first em- 
bodiment of the invention. 

Fig. 5 is an explanatory diagram showing an edit- 
ing method of compressed picture data in a second 
embodiment of the invention. 

Fig. 6 is an explanatory diagram showing an edit- 
ing method of compressed picture data in a third em- 
bodiment of the invention. 

Fig. 7 is an explanatory diagram showing an edit- 
ing method of compressed picture data in the third 
embodiment of the invention. 

Fig. 8 is a block diagram of a picture mixing ap- 
paratus in a fourth embodiment of the invention. 

Fig. 9 is an explanatory diagram of algorithm 
showing an example of operation of CPU in the fourth 
embodiment of the invention. 

Fig. 10 is an explanatory diagram showing a mo- 
tion compensation method of mixed picture in the 
fourth embodiment of the invention. 

Fig. 11 is a block diagram of a picture mixing ap- 
paratus in a fifth embodiment of the invention. 

Fig. 12 is an explanatory diagram of a com- 
pressed picture mixing method in a sixth embodiment 



of the invention. 

Fig. 13 is an explanatory diagram of a com- 
pressed picture mixing apparatus in the sixth embodi- 
ment of the invention. 
5 Fig. 14 is a block diagram of the picture mixing 

apparatus in the sixth embodiment of the invention. 

Fig. 15 is a block diagram of a conventional pic- 
ture encoder. 

Fig. 16 is an explanatory diagram of a conven- 
10 tional motion compensation prediction method. 

Fig. 1 7 is a constitution diagram of conventional 
motion compensation circuit and frame memory. 

Fig. 18 is an explanatory diagram showing a con- 
ventional editing method of compressed picture data. 

15 

DESCRIPTION OF THE PREFERRED 
EMBODIMENTS 

A compressed picture editing method according 

20 to a first embodiment of the invention is described be- 
low while referring to the drawings. 

Fig. 1 is a block diagram of a picture encoder in 
the first embodiment of the invention. In Fig. 1 , refer- 
ence numeral 1 denotes a picture decoding unit which 

25 comprises a variable length decoding circuit 11 , an in- 
verse quantizing circuit 1 2, an inverse DCT circuit 1 3, 
a frame memory 14, and a simplified motion compen- 
sation circuit 1 5. Reference numeral 2 is a picture en- 
coding unit which comprises a DCT circuit 21 , a quan- 

30 tizing circuit 22, a variable length coding circuit 23, an 
inverse quantizing circuit 24, an inverse DCT circuit 
25, a frame memory 26, and a simplified motion com- 
pensation circuit 27. Reference numeral 3 is a control 
unit which comprises a CPU. Fig. 2 is a detailed struc- 

35 tural diagram of the simplified motion compensation 
circuit 15 and frame memory 14, Fig. 3 is an explana- 
tory diagram of algorithm showing an example of op- 
eration of the CPU 3, and Fig. 4 is an explanatory di- 
agram showing an editing method of compressed pic- 

40 ture data. The frame accompanied by an apostrophe 
(') denotes a re-coded frame. It is the same in Figs. 5 
through 7. 

Thus constituted picture encoder is described 
while referring to Figs. 1 through 4. 

45 Suppose to link compressed picture data (a) and 

(b) in Fig. 4. For the sake of simplicity, the editing 
point is supposed to be a P frame. When the com- 
pressed picture data (a) in Fig. 4 enters the picture 
encoder in Fig. 1, it is put out as it is until coming to 

so the editing point indicated by the dotted line in Fig. 4. 
On the other hand, the compressed picture data (b) 
in Fig. 4 is put in the picture decoding unit 1 , but the 
output of the picture coding unit 2 is not sent outside 
up to the editing point. The compressed picture data 

55 (b) is decoded in the picture decoding unit 1 . That is, 
depending on the DCT mode information that is de- 
coded in variable length in the variable length decod- 
ing circuit 11 , inversely quantized in the inverse quan- 
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tizing circuit 12, and decoded in the inverse OCT cir- 
cuit 13, inverse DCT is performed on a frame by 
frame or field by field basis to return to real time pic- 
ture data. Yet, because of the difference coding, us- 
ing the decoded motion vector and motion comperv 5 
sation mode information, a predicted picture is gen- 
erated in the frame memory 14 and simplified motion 
compensation circuit 15, and is summed with the in- 
verse DCT circuit output data, thereby creating de- 
coded picture data. Fig. 2 is a constitutional example 10 
of the frame memory 14 and simplified motion com- 
pensation circuit 15. As compared with the conven- 
tional picture encoder, the constitution of the frame 
memory 14 is the same, but the motion compensation 
circuit 15 is substantially different Since the motion is 
compensation mode has been already determined by 
the encoder, any square error calculation circuit is not 
needed in the picture decoding unit, and hence it only 
requires, as shown in Fig. 2, a mean calculating circuit 
151 necessary when the bidirectional prediction is se- 20 
lected, and a selector 152 for issuing a predicted pic- 
ture depending on the motion compensation mode in- 
formation. 

When a signal showing an editing point is en- 
tered, as shown in (c), the P frame right after the edit- 25 
ing point, among the compressed picture data (b), is 
re-coded as an I frame, and is linked to the com- 
pressed picture data (a) and issued. In the com- 
pressed picture data (b), the B frames of frame num- 
bers 6, 7 are coded again by backward prediction be- 30 
cause the I frame of frame number 5 necessary for 
forward prediction has been lost by editing. As for the 
P frame after frame number 8 in the compressed pic- 
ture data (b), since the picture used in prediction has 
been coded again, the predicted picture is set again 35 
to a correct one, and the P frame is coded again into 
a new P frame, and the B frame B into a new B frame. 
The re-coding method is nearly the same as in the pri- 
or art, except that the motion compensation mode in- 
formation, motion vector, and DCT changeover infor- 40 
mation are obtained by decoding the compressed pic- 
ture data (b). Therefore, the picture coding unit 2 does 
not require a motion detection circuit and a DCT mode 
judging circuit for mass calculation as used in the con- 
ventional picture encoder in Fig. 14, and the motion 45 
compensation circuit 89 can be replaced by the sim- 
plified motion compensation circuit 27 same as the 
one in the decoding apparatus. All of the control is ef- 
fected by the CPU 3, and its algorithm is shown in Fig. 
3. 50 

Fig. 3 shows a flow chart of the main routine 
showing the entire operation, sub-routine for coding 
the P frame right after the editing point used in the 
main routine into an I frame, sub-routine for coding 
the B frame into a P frame by using backward predic- 55 
tion only, sub-routine for re-coding the P frame and 
B frame, and intraframe coded block coding sub-rou- 
tine and backward prediction block coding sub-rou- 



tine for coding by changing the motion compensation 
mode of block of each frame being used in the fore- 
going sub-routines. 

In a general decoder, a reception buffer of a spe- 
cific capacity is provided, and deviation of transmis- 
sion rate and decoding speed of the decoder is absor- 
bed. In the encoder, compression and coding are ef- 
fected while controlling so that the reception buffer of 
the decoder may cause neither overflow nor under- 
flow. Similar operation is needed when editing com- 
pressed picture data. In this embodiment, for exam- 
ple, the CPU 3 sequentially calculates the buffer con- 
tent in the reception buffer, and the quantizing width 
of the quantizing circuit 22 is controlled if necessary 
so that the compressed picture data after editing may 
cause neither overflow nor underflow of the reception 
buffer. Besides, as a simpler method for preventing 
overflow at the editing point, it is also possible to add 
dummy data other than picture data (all zero, etc.) to 
the picture data. 

Thus, according to the embodiment, since the 
compressed image data to be edited is re-coded in 
the picture encoder of a simple constitution, even the 
compressed picture data using interframe difference 
coding can be edited without losing the compressed 
picture data after splicing. 

Fig. 5 is an explanatory diagram of an editing 
method of picture compressed data in a second em- 
bodiment of the invention. The picture compressed 
data of (b) in Fig. 5 contains an I frame after the edit- 
ing point. From the editing point to the point before the 
frame number 11, the frames are re-coded in the 
same manner as in the first embodiment. Since the I 
frame of frame number 11 has been coded within the 
frame, it is not affected by editing. Therefore, the I 
frame is not re-coded. As for the B frames of frame 
numbers 9, 10, since the P frame of frame number 8 
is coded newly into an I frame, the B frames 9, 1 0 are 
re-coded by using the newly coded I frame. As for the 
frames after frame number 14, since the I frame of 
frame number 11 is not re-coded, there is no effect 
and it is not necessary to re-code, and the com- 
pressed picture data is issued as it is. The operation 
so far can be realized by varying the program of the 
CPU 3 in the first embodiment. 

According to the embodiment, thus, the number 
of frames to be re-coded may be decreased, and pic- 
ture quality deterioration due to re-coding may be 
kept to a minimum limit. 

In the second embodiment, frames after frame 
number 14 are not re-coded because the I frame of 
frame number 11 is not re-coded. However, as men- 
tioned in the first embodiment, when control of recep- 
tion buffer is necessary, not only the B frame right af- 
ter the I frame, but also all P and B frames may be re- 
coded as shown in (d) of Fig. 5. 

According to this technique, while keeping few 
the frames to be re-coded, compressed picture data 
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after editing free from problems of overflow or under- 
flow of reception buffer may be obtained. 

Fig. 6 and Fig. 7 are explanatory diagrams of edit- 
ing method of picture compressed data in a third em- 
bodiment of the invention. In this embodiment, the 
editing point in (b) of Fig. 6 of the compressed picture 
data to be connected is in a B frame. When the editing 
point is selected in the B frame, the editing point and 
the B frame continuous to the editing point cannot be 
decoded because the I or P frame used in prediction 
is lost at the time of coding in the first and second em- 
bodiments. In this embodiment, as shown in (c) of Fig. 
6 or (c) of Fig. 7, an I or P frame just before the editing 
point used in prediction is inserted by the number of 
undecodable B frames. When the frame just before 
the editing point used in prediction is an I frame, as 
shown in (c) of Fig. 6, this I frame just before the edit- 
ing point is inserted as it is, basically. In the case of 
a P frame, it is re-coded into an I frame as shown in 
(c) of Fig. 7. If the reception buffer is a problem, same 
as in the second embodiment, even in the case of the 
I frame, it is possible to re-code to control the quan- 
tizing width so as not to cause overflow or underflow. 

Thus, according to the embodiment, if the editing 
point of the compressed picture data to be connected 
is a B frame, the picture quality deterioration due to 
re-coding may be kept to a minimum, while editing is 
enabled. 

In the foregoing embodiments, there are two B 
frames, but this is not limited and it may be applied to 
three or more or one B frame. 

In the embodiments above, for the sake of sim- 
plicity, the intraf rame coding block in motion compen- 
sation mode of each block is omitted in explanation, 
but generally intraframe coding block mode can be 
selected on a two-dimensional block by block basis in 
the frame. When changing a B frame into a P frame 
composed only of backward prediction, the motion 
compensation mode of forward prediction block or the 
like can be re-coded in the same motion compensa- 
tion mode as the motion vector is not contained in the 
compressed picture data. In such a case, it can be re- 
coded by selecting an intraframe coding block. 

A compressed picture mixing method and a com- 
pressed picture mixing apparatus in a fourth embodi- 
ment of the invention are described below while refer- 
ring to the accompanying drawings. 

Fig. 8 is a block diagram of a picture mixing ap- 
paratus in the fourth embodiment of the invention. In 
Fig. 8, reference numeral 1 denotes a picture decod- 
ing unit, and 2 is a picture coding unit, and they oper- 
ate in the same way as in the first embodiment. Fig.9 
is an explanatory diagram of algorithm showing an ex- 
ample of operation of CPU 3, and Fig. 10 is an expla- 
natory diagram showing the motion compensation 
method of mixed picture. 

Thus constituted picture encoder is described 
below by reference to Figs. 8 through 10. 



Suppose to mix a picture data with a compressed 
picture data that has been prediction-coded. The 
compressed picture data in Fig. 8 is entered from an 
input terminal 16, while a certain picture data is fed 
5 from an input terminal 41. The compressed picture 
data is decoded in the picture decoding unit 1 in the 
same way as in the first embodiment. 

On the other hand, from the picture data to be 
mixed, picture information is detected in a picture in- 
10 formation detector 4. The picture data to be mixed 
may be either an uncompressed picture data or an- 
other compressed picture data. The picture informa- 
tion detector 4 is a circuit for detecting the picture in- 
formation to be used in coding a mixed picture. It may 
15 be omitted if it is not necessary to enhance the coding 
efficiency. The picture information to be detected 
may include, in the case of uncompressed picture 
data, motion information showing how may pixels in 
the picture are moving between frames. In the com- 
20 pressed picture data, the picture information detector 
may be composed in the same way as the picture de- 
coding circuit 1, and the information to be detected 
may include motion vector, motion compensation in- 
formation, and orthogonal transform mode informa- 
25 tion. 

Mixing is achieved by adding the picture data de- 
coded by the picture decoding unit 1 and the picture 
data output-ted from the picture information detector 
4. In principle, mixing can be started from any frame, 

30 but it is desired to start from an intraframe coded 
frame (I frame). The reason is that, if started from a 
predicted frame, a completely new picture appears as 
if the scene were changed, which cannot be predicted 
from the preceding frame, thereby lowering the cod- 

35 ing efficiency. Besides, depending on the picture to 
be mixed, the content is not changed frequently such 
as the superimposed dialogue. In such a case, by 
changing the content at intervals of the intraframe 
coding frames, the interframe difference can be de- 

40 creased, so that the coding efficiency can be en- 
hanced. 

The mixed picture data after addition is re-coded 
in the picture coding unit 2 into compressed picture 
data. Fig. 10 shows the motion compensation method 
45 of blocks in each frame. The solid line in Fig. 10 de- 
notes decoded picture data, and picture data of brok- 
en line, or block b, is the added mixed picture data. 
Frame t+3 uses frame t as prediction picture. The de- 
coded motion vectors of the respective blocks are in- 
50 dicated by arrows mva, mvb, mvc in the diagram. 

In block a, decoded picture data is used as pre- 
diction picture. The re-coding method of block a is 
nearly the same as the coding method in the prior art, 
but the motion compensation mode information, mo- 
55 tion vector, and DCT change-over information are in- 
formation obtained by "decoding the compressed pic- 
ture data. Therefore, in the picture coding unit 2, a 
motion detector and a DCT mode judging circuit re- 
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quiring massive calculation are eliminated differently 
from the image encoder of the prior art shown in Fig. 
16, and the motion compensation circuit 89 may be 
replaced with a simplified motion compensation cir- 
cuit 27 which is the same as the one in the decoding 5 
apparatus. 

Block b is replaced with mixed picture data in 
frame t+3. Therefore, the content is completely differ- 
ent from the predicted image generated in the motion 
vector mvb. Therefore, by intraf rame coding the block 10 
b, or replacing mvb with mvb', the motion compensa- 
tion information and motion vector are changed so as 
to predict from the mixed block. Herein, mvb' is the 
motion information of the picture data to be mixed as 
being detected in the picture information detector 4. 15 

Block c is predicted by motion vector mvc, but the 
predicted picture of frame t is replaced by mixing, and 
therefore the predicted picture is completely differ- 
ent. In block c, hence, by replacing the intraframe 
coding block or mvc with mvc', the motion compensa- 20 
tion and motion vector are changed so as to predict 
from the decoded picture data. 

It is the CPU 3 that is responsible for these con- 
trols, and Fig. 9 shows an algorithm of an example of 
operation of the CPU 3. In Fig. 9, of the operation de- 25 
scribed above, the case of changing the block lost of 
predicted picture into an intraframe coding block is 
sequentially shown in flow chart 

On the above embodiment, the compressed pic- 
ture data is decoded and added to another picture 30 
data, and the mixed picture data is re-coded by using 
the motion compensation information or the like ob- 
tained by decoding the compressed picture data, so 
that the compressed mixed picture data can be ob- 
tained in a simple constitution. 35 

Moreover, the predicted picture lost by mixing is 
replaced by a new predicted picture obtained by intra- 
frame coding or vector correction, and the picture in 
the region is predicted by using the motion informa- 
tion of the mixed picture, so that mismatching with the 40 
predicted picture does not occur, and efficient coding 
is effected without causing picture deterioration. 

In the foregoing embodiments, the picture is 
mixed on a block by block basis, but this is not limit- 
ative, and mixing over plural blocks is also possible. 45 
In such a case, to change the motion compensation 
information, the motion compensation information of 
decoded picture data or motion information of picture 
data to be mixed may be selectively used depending 
on the rate of mixed data contained in the block. so 

In the above embodiments, as the picture coding 
unit and picture decoding unit, the interframe differ- 
ence and DCTare combined, but this is not limitative, 
and any other method may be employed as far as the 
motion compensation and interframe difference are 55 
employed. 

Fig. 11 is a block diagram of a compressed pic- 
ture mixing apparatus in a fifth embodiment of the in- 



vention. What differs from the fourth embodiment is 
that orthogonal transform coefficients are added as 
the manner of adding the picture data. That is, instead 
of directly adding the pixels, the pixels are converted 
into coefficients by orthogonal transform, and the 
transform coefficients are added. Since, the picture 
data to be added are transform coefficients, a trans- 
form circuit 42 is provided in the picture information 
detector 4 in Fig. 11. The transform circuit may be. 
simply, only a DCT circuit for transforming the pixel 
data into transform coefficients. Alternatively, using 
the same coding circuit as in the prior art, the trans- 
form coefficients after DCT transform may be issued. 

In this embodiment, too, the same effect as in the 
fourth embodiment may be obtained. 

Fig. 12 is an explanatory diagram of compressed 
picture mixing method in a sixth embodiment of the 
invention. In this embodiment, as shown in Fig. 12, 
compressed picture data A and compressed picture 
data B to be mixed with the compressed picture data 
A are multiplexed by giving different identifiers. Fig. 
1 3 shows an example of a circuit for generating the 
multiplexed compressed picture data shown in Fig. 
12. In Fig. 13, the compressed picture data A is stored 
in a first buffer 131, and the compressed picture data 
B is stored in a second buffer 132. A multiplexer 134 
adds the identifier generated in an identifier generat- 
ing circuit 132 before each compressed picture data 
while adjusting the timing, and multiplexes and deliv- 
ers the data in the sequence in Fig. 12. 

Fig. 14 is a block diagram of a picture mixing ap- 
paratus for decoding and mixing the multiplexed com- 
pressed picture data. The multiplexed compressed 
picture data are separated depending on the identifi- 
er by a de-multiplex circuit 5, and fed into different 
picture decoding units 1. Each picture decoding unit 
1 is the same as the picture decoding unit 1 in Fig. 11. 
The picture data obtained by decoding in the picture 
decoding units are added by an adder 140 to generate 
a mixed picture data. 

In this way, by multiplexing two or more com- 
pressed picture data to be mixed, separating them at 
the decoding side, and adding the pixel data thereof, 
a mixed picture is obtained. In this embodiment, plu- 
ral picture decoding units 1 are necessary, but for ex- 
ample, in the case of simultaneous broadcasting of 
multiple programs, when it is desired to mix same 
character information in each, the mixing devices are 
needed as many as the number of programs at the 
transmission side in embodiments 4 and 5, but only 
one is enough at the transmission side in this embodi- 
ment. Or, by selecting the identifier at the reception 
side, it is also possible to select the picture to be 
mixed. 

The sixth embodiment is an example of com- 
pressed picture data being compressed by prediction 
and coding, but this is not limitative, and it can be also 
applied to compressed picture data using only intra- 
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frame coding. 

Moreover in the sixth embodiment, having two 
picture decoding units, the pictures decoded by the 
respective picture decoding units are added, but de- 
pending on the method of mixing pictures, it is possi- 
ble to replace a part of one picture with a part of other 
picture. In such a case, two picture decoding units are 
not needed, and by setting picture decoding in the 
same picture decoding unit at different times, a mixed 
picture may be obtained by a single picture decoding 
unit. 

In the foregoing embodiments, nothing is men- 
tioned about the resolution of the mixing picture and 
mixed picture, but these pictures may differ in reso- 
lution, and a small picture may be mixed in a part of 
a large picture, or only a partial region of a picture 
may be mixed if identical in resolution. 



Claims 

1 . A compressed picture editing method for linking 
at an editing point first and second compressed 
picture data each having been obtained by pre- 
diction-coding employing at least one of a for- 
ward prediction for predicting a picture data in a 
frame from a preceding frame, a backward pre- 
diction for predicting from a succeeding frame 
and a bidirectional prediction using both the for- 
ward prediction and backward prediction, said 
method comprising the steps of: 

receiving the first and second compressed 
picture data; 

decoding a frame of the second com- 
pressed picture data at a position corresponding 
to the editing point to generate a decoded frame; 

intraframe coding the decoded frame to 
generate a modified second compressed picture 
data having an intraframe coded frame at the 
position corresponding to the editing point; and 

linking the first compressed picture data 
and the modified second compressed picture 
data at the editing point. 

2. A method according to claim 1 , wherein said de- 
coding step produces motion compensation in- 
formation of the decoded frame, and wherein 
said method further comprising the steps of de- 
coding at least a part of the remaining frames of 
the second compressed picture data to obtain de- 
coded frames, and prediction-coding the decod- 
ed frames by using the motion compensation in- 
formation. 

3. A method according to claim 1 , wherein said de- 
coding step produces motion vector information 
of the decoded frame, and wherein said method 
further comprises the steps of decoding at least 



a part of the remaining frames of the second com- 
pressed picture data to obtain decoded frames, 
and prediction-coding the decoded frames by us- 
ing the motion vector information. 

5 

4. A method according to claim 1, wherein said 
method further comprises the steps of decoding 
at least a part of the remaining frames of the sec- 
ond compressed picture data to obtain decoded 

10 frames, and prediction-coding the decoded 

frames so as to change a bidirectional prediction 
frame in the second compressed picture data to 
either a forward prediction frame or a backward 
prediction frame. 

15 

5. A method according to claim 1, wherein said 
method further comprises the steps of decoding 
at least a part of the remaining frames of the sec- 
ond compressed picture data to obtain decoded 

20 frames, and prediction-coding the decoded 

frames except for a part or all of intraframe coded 
frames in the second compressed picture data. 

6. A method according to claim 1, wherein said 
25 method further comprises the steps of decoding 

at least a part of the remaining frames of the sec- 
ond compressed picture data to obtain decoded 
frames, and prediction-coding the decoded 
frames so as to change a part or all of a forward 
30 prediction frame, a backward prediction frame 

and a bidirectional prediction frame in the second 
compressed picture data to an intraframe coded 
frame. 

35 7. A method according to daim 1, wherein said 
method further comprises the steps of decoding 
at least a part of the remaining frames of the sec- 
ond compressed picture data to obtain decoded 
frames, and prediction-coding the decoded 

40 frames so as to change a bidirectional prediction 

frame in the second compressed picture data to 
an intraframe coded frame which corresponds to 
an intraframe coded frame which has been used 
for predicting the bidirectional frame. 

45 

8. A method according to claim 1, wherein said 
method further comprises the steps of decoding 
at least a part of the remaining frames of the sec- 
ond compressed picture data to obtain decoded 
so frames, and prediction-coding the decoded 

frames so as to control the amount of coded data 
to prevent overflow and underflow of a buffer in 
a decoding apparatus. 

55 9. A method according to claim 1, wherein said 
method further comprises the steps of decoding 
at least a part of the remaining frames of the sec- 
ond compressed picture data to obtain decoded 
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frames, prediction-coding the decoded frames to 
obtain re-coded frames, and adding a dummy 
data to the re-coded frames. 

10. A compressed picture editing method for linking 5 
at an editing point first and second compressed 
picture data each having been obtained by divid- 
ing picture data in each frame into a plurality of 
two-dimensional blocks and coding each of the 
two-dimensional blocks by selectively using one 10 
of a frame-based orthogonal transform and a 
field-based orthogonal transform, said method 
comprising the steps of: 

receiving the first and second compressed 
picture data; 15 

decoding a frame of the second com- 
pressed picture data at a position around the edit- 
ing point to generate a decoded frame and to ob- 
tain orthogonal transform mode information indi- 
cating whether each two-dimensional block in 20 
the decoded frame has been subjected to the 
frame-based orthogonal transform or the field- 
based orthogonal transform; 

coding the decoded frame in accordance 
with the orthogonal transform mode information 25 
to obtain a re-coded frame and to generate a 
modified second compressed picture data con- 
taining the re-coded frame; and 

linking the first compressed picture data 
and the modified second compressed picture 30 
data at the editing point. 

11. A compressed picture editing apparatus for link- 
ing at an editing point first and second com- 
pressed picture data each having been obtained 35 
by prediction-coding employing at least one of a 
forward prediction for predicting a picture data in 

a frame from a preceding frame, a backward pre- 
diction for predicting from a succeeding frame 
and a bidirectional prediction using both the for- 40 
ward prediction and backward prediction, said 
apparatus comprising: 

means for receiving the first and second 
compressed picture data; 

means for decoding a frame of the second 45 
compressed picture data at a position corre- 
sponding to the editing point to generate a decod- 
ed frame; 

means for intraf rame coding the decoded 
frame to generate a modified second com- 50 
pressed picture data having an intraframe coded 
frame at the position corresponding to the editing 
point; and 

means for linking the first compressed pic- 
ture data and the modified second compressed 55 
picture data at the editing point 

12. A compressed picture editing apparatus for link- 



ing at an editing point first and second com- 
pressed picture data each having been obtained 
by dividing picture data in each frame into a plur- 
ality of two-dimensional blocks, transforming 
each of the two-dimensional blocks by selective- 
ly using one of a frame-based orthogonal trans- 
form and a field-based orthogonal transform, and 
prediction-coding each transformed two-dimen- 
sional block by employing at least one of a for- 
ward prediction for predicting a picture data in a 
frame from a preceding frame, a backward pre- 
diction for predicting from a succeeding frame 
and a bidirectional prediction using both the for- 
ward prediction and backward prediction, said 
apparatus comprising: 

means for receiving the first and second 
compressed picture data; 

decoding means for decoding a frame of 
the second compressed picture data at a position 
around the editing point to generate a decoded 
frame and to obtain motion compensation infor- 
mation and motion vector information of the de- 
coded frame and orthogonal transform mode in- 
formation indicating whether each two-dimen- 
sional block in the decoded frame has been sub- 
jected to the frame-based orthogonal transform 
or the field-based orthogonal transform; 

encoding means for coding the decoded 
frame by using at least one of the motion com- 
pensation information, motion vector information 
and orthogonal transform mode information to 
obtain a re-coded frame and to generate a modi- 
fied second compressed picture data containing 
the re-coded frame; 

control means receiving the motion com- 
pensation information, motion vector information 
and orthogonal transform mode information for 
controlling the encoding means; and 

means for linking the first compressed pic- 
ture data and the modified second compressed 
picture data at the editing point. 

13. A compressed picture mixing method for mixing 
a first picture data and a second picture data, the 
first picture data being a compressed picture 
data which has been obtained by prediction-cod- 
ing employing at least one of a forward prediction 
for predicting a picture data in a frame from a pre- 
ceding frame, a backward prediction for predict- 
ing from a succeeding frame and a bidirectional 
prediction using both the forward prediction and 
backward prediction, said method comprising the 
steps of: 

receiving the first and second picture data; 

decoding the first picture data to obtain a 
decoded first picture data and motion compensa- 
tion information of the first picture data; 

adding the decoded first picture data and 
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the second picture data to obtain a mixed picture 
data; and 

coding the mixed picture data by using the 
motion compensation information. 

5 

14. A compressed picture mixing method for mixing 
a first picture data and a second picture data, the 
first picture data being a compressed picture 
data which has been obtained by prediction-cod- 
ing employing at least one of a forward prediction 10 
for predicting a picture data in a frame from a pre- 
ceding frame, a backward prediction for predict- 
ing from a succeeding frame and a bidirectional 
prediction using both the forward prediction and 
backward prediction, said method comprising the 15 
steps of: 

receiving the first and second picture data; 

decoding the first picture data to obtain 
first orthogonal transform coefficients of the first 
picture data and motion compensation informa- 20 
tion of the first picture data; 

orthogonal transform coding the second 
picture data to obtain second orthogonal trans- 
form coefficients; 

adding the first orthogonal transform coef- 25 
ficients and the second orthogonal transform 
coefficients to obtain mixed orthogonal trans- 
form coefficients; 

coding the mixed orthogonal transform 
coefficients by using the motion compensation 30 
information. 

15. A compressed picture mixing method for mixing 
a first picture data and a second picture data, the 
first picture data being a compressed picture 35 
data which has been obtained by prediction-cod- 
ing employing at least one of a forward prediction 
for predicting a picture data in a frame from a pre- 
ceding frame, a backward prediction for predict- 
ing from a succeeding frame and a bidirectional 40 
prediction using both the forward prediction and 
backward prediction, said method comprising the 
steps of: 

receiving the first and second picture data; 

decoding the first picture data to obtain a 45 
decoded first picture data; 

adding at least a part of the decoded first 
picture data and the second picture data to obtain 
a mixed picture data which includes a first part 
containing only the decoded first picture data and 50 
a second part containing only the second picture 
data; and 

coding the mixed picture data, said coding 
step coding said first part by prediction using only 
a picture data contained in the first picture data 55 
and coding said second part by prediction using 
only a picture data contained in the second pic- 
ture data. 



16. A compressed picture mixing method for mixing 
a first picture data and a second picture data, the 
first picture data being a compressed picture 
data which has been obtained by prediction-cod- 
ing employing at least one of a forward prediction 
for predicting a picture data in a frame from a pre- 
ceding frame, a backward prediction for predict- 
ing from a succeeding frame and a bidirectional 
prediction using both the forward prediction and 
backward prediction, said method comprising the 
steps of: 

receiving the first and second picture data; 

decoding the first picture data to obtain a 
decoded first picture data; 

adding at least a part of the decoded f irst 
picture data and the second picture data to obtain 
a mixed picture data which includes a part con- 
taining the second picture data; and 

coding the mixed picture data, said coding 
step including a step of motion compensating 
said part by using motion compensation informa- 
tion derived from one of the first picture data and 
the second picture data. 

17. A compressed picture mixing method for mixing 
a f irst picture data and a second picture data, the 
first picture data being a compressed picture 
data which has been obtained by dividing picture 
data in each frame into a plurality of two-dimen- 
sional blocks and coding each of the two-dimen- 
sional blocks by selectively using one of a frame- 
based orthogonal transform and a field-based or- 
thogonal transform, said method comprising the 
steps of: 

receiving the first and second compressed 
picture data; 

decoding the first picture data to obtain a 
decoded first picture data and orthogonal trans- 
form mode information indicating whether each 
two-dimensional block in the decoded frame has 
been subjected to the frame-based orthogonal 
transform or the field-based orthogonal trans- 
form; 

adding the decoded first picture data and 
the second picture data to obtain a mixed picture 
data; and 

coding the mixed picture data by using the 
orthogonal transform mode information. 

18. A compressed picture mixing method for mixing 
a first picture data and a second picture data, the 
first picture data being a compressed picture 
data which has been obtained by prediction-cod- 
ing employing at least one of a forward prediction 
for predicting a picture data in a frame from a pre- 
ceding frame, a backward prediction for predict- 
ing from a succeeding frame and a bidirectional 
prediction using both the forward prediction and 
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backward prediction, said method comprising the 
steps of: 

receiving the first and second picture data; 

and 

adding the first picture data and the sec- 5 
ond picture data from a position where the first 
picture data contains an intraf rame coded frame 
to obtain a mixed picture data. 

19. A compressed picture mixing method for mixing jo 
a first picture data and a second picture data, the 

first picture data being a compressed picture 
data which has been obtained by prediction-cod- 
ing employing at least one of a forward prediction 
for predicting a picture data in a frame from a pre- 15 
ceding frame, a backward prediction for predict- 
ing from a succeeding frame and a bidirectional 
prediction using both the forward prediction and 
backward prediction, said method comprising the 
steps of: 20 
receiving the first and second picture data; 

and 

adding the first picture data and the sec- 
ond picture data to obtain a mixed picture data, 
said adding step including a step of changing the 25 
second picture data by using an intraframe cod- 
ed frame contained in the first picture data. 

20. A compressed picture mixing apparatus for mix- 
ing a first picture data and a second picture data, 30 
the first picture data being a compressed picture 
data which has been obtained by dividing picture 
data in each frame into a plurality of two-dimen- 
sional blocks, transforming each of the two-di- 
mensional blocks by selectively using one of a 35 
frame-based orthogonal transform and a field- 
based orthogonal transform, and prediction-cod- 
ing each transform two-dimensional block by em- 
ploying at least one of a forward prediction for 
predicting a picture data in a frame from a pre- 40 
ceding frame, a backward prediction for predict- 
ing from a succeeding frame and a bidirectional 
prediction using both the forward prediction and 
backward prediction, said apparatus comprising: 

means for receiving the first and second 45 
picture data; 

decoding means for decoding the first pic- 
ture data to obtain a decoded first picture data 
and to obtain motion compensation information 
and motion vector information of the first picture 50 
data and orthogonal transform mode information 
indicating whether each two-dimensional block in 
each frame of the first picture data has been sub- 
jected to the frame-based orthogonal transform 
or the field-based orthogonal transform; 55 

adding means for adding the decoded first 
picture data and the second picture data to obtain 
a mixed picture data; 



encoding means for coding the mixed pic- 
ture data by using at least one of the motion com- 
pensation information, motion vector information 
and orthogonal transform mode information; and 

control means receiving the motion com- 
pensation information, motion vector information 
and orthogonal transform mode information for 
controlling the encoding means. 

21. A compressed picture mixing apparatus for mix- 
ing a first picture data and a second picture data, 
the first picture data being a compressed picture 
data which has been obtained by dividing picture 
data in each frame into a plurality of two-dimen- 
sional blocks, transforming each of the two-di- 
mensional blocks by selectively using one of a 
frame-based orthogonal transform and a field- 
based orthogonal transform, and prediction-cod- 
ing each transformed two-dimensional block by 
employing at least one of a forward prediction for 
predicting a picture data in a frame from a pre- 
ceding frame, a backward prediction for predict- 
ing from a succeeding frame and a bidirectional 
prediction using both the forward prediction and 
backward prediction, said apparatus comprising: 

means for receiving the first and second 
picture data; 

decoding means for decoding the first pic- 
ture data to obtain first orthogonal transform 
coefficients and to obtain motion compensation 
information and motion vector information of the 
first picture data and orthogonal transform mode 
information indicating whether each two-dimen- 
sional block in each frame of the first picture data 
has been subjected to the frame-based orthogo- 
nal transform or the field-based orthogonal 
transform; 

transform means for orthogonal trans- 
forming the second picture data to obtain second 
orthogonal transform coefficients; 

adding means for adding the first orthogo- 
nal transform coefficients and the second ortho- 
gonal transform coefficients to obtain mixed or- 
thogonal transform coefficients; 

encoding means for coding the mixed or- 
thogonal transform coefficients by using at least 
one of the motion compensation information, mo- 
tion vector information and orthogonal transform 
mode information; and 

control means receiving the motion com- 
pensation information, motion vector information 
and orthogonal transform mode information for 
controlling the encoding means. 

22. A picture coding method comprising the steps of: 
coding a first picture data to obtain a first 
compressed picture data; 

coding a second picture data which is to be 
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mixed with the first picture data to obtain a sec- 
ond compressed picture data; 

adding a first identifier indicating the first 
picture data to the first compressed picture data 
and adding a second identifier indicating the sec- 5 
ond picture data to the second compressed pic- 
ture data; 

multiplexing the first compressed picture 
data added with the first identif ierand the second 
compressed picture data added with the second io 
identifier. 

23. A compressed picture data mixing-apparatus for 
mixing a first compressed picture data and a sec- 
ond compressed picture data which have been is 
transmitted from an encoder in the form of a mul- 
tiplexed data in which the first compressed pic- 
ture data added with a first identifier and the sec- 
ond compressed picture data added with a sec- 
ond identifier differentf rom the first identifier are 20 
multiplexed, said apparatus comprising: 

means for receiving the multiplexed data; 

means for extracting the first and second 
identifiers from the received multiplexed data 
and for extracting from the received multiplexed 25 
data the first compressed picture data and the 
second compressed picture data based on the 
extracted first and second identifiers; 

decoding means for decoding the extract- 
ed first and second compressed picture data in- 30 
dependently of each other to obtain first and sec- 
ond decoded picture data, respectively; and 

adding the first decoded picture data and 
the second decoded picture data to obtain a 
mixed picture data. 35 
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